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the TSP evaluation in 1988 were no longer a proper comparison group, 
evaluators selected a new set of 12 comparison schools. The mission of the 
TSP is to provide an instructional program and an organizational design that 
is language instruction intensive to reverse the pattern of poor academic 
achievement of African American and other students in schools with 
predominantly minority populations. Analyses using 1997-1998 and 1998-1999 
data support the sampling decision that the TSP schools and newly selected 
comparison schools serve the same population. Analyses indicate that TSP 
students outperformed comparison students in reading, mathematics, and 
language at a p<0.001 level of significance. The effect size for the TSP 
program exceeds the corresponding comparison schools' effect size in all 
three areas, especially in mathematics and language. After adjusting for 
pre-existing differences between TSP schools and comparison schools, there 
are still statistically significant differences in each subject area tested. 
These differences are especially meaningful for mathematics and language 
achievement tests. An appendix contains a reference guide to the TSP program. 
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Executive Summary 



Evaluation Objective 

The main objective of this evaluation was to compare the achievement of the Ten 
Schools Program (TSP) students 1 with the achievement of students in comparable 
schools. Since the original 10 comparison schools selected for the evaluation of the Ten 
Schools Program in 1988 were no longer a proper comparison group, the Program 
Evaluation and Research Branch (PERB) selected a new set of 1 2 comparison schools 
based on a number of criteria including student academic achievement level, school's 
ethnic mix, and school's geographical location. 

Evaluation Questions 

This evaluation attempts to answer the following evaluation questions: 

1 . Do the newly selected comparison schools and schools included in TSP derive 
from the same population? In other words, do the newly selected comparison 
schools match the TSP schools based on their students’ academic achievement 
and other historically significant factors that impact students' academic 
achievement? 

2. Is there a significant difference between the TSP students' level of 
achievement gain and the achievement gain of students in the selected 
comparison schools? 

Results 

1 . Analyses using 1 997-98 and 1 998-99 data support our sampling decision that 
the TSP schools and newly selected comparison schools belong to the same 
population. 

2. Based on the results of our analyses TSP students outperformed comparison 
schools students in the areas of reading, math and language at p<0.001 level 
of significance. The effect size for the TSP program exceeds the 
corresponding comparison schools’ effect size in all three areas, but especially 
in math and language. 



1 Actually 12 schools since 2 more schools were added to the list of Schools in TSP in 1998. 
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3. After adjusting for the pre-existing differences between TSP schools and 
comparison schools, there still are statistically significant differences in each 
subject area tested. TSP students outperform students in comparison schools 
in all three areas tested. 

Conclusions 

The schools selected for comparison in 1998-99 and the Ten Schools program are 
drawn from the same population. They were comparable in 1997-98 and 1998-99 not 
only in terms of their level of achievement but also on some crucial background variables 
such as percent of Title 1 participants, percent of students receiving free/reduced lunch, 
and percent of Limited English Proficient (LEP) students. 

The TSP schools students outperformed the comparison schools in the areas of 
reading, mathematics, and language. These differences are statistically significant, and 
educationally meaningful especially for mathematics and language achievement tests. 




vi 



Comparative Analyses of Students Academic Achievement in Ten Schools Program and 

Selected Comparison Schools 

Introduction 

The Ten Schools Program (TSP) is a research-based instructional and organizational 
program, initiated by the Los Angeles Unified School District (LAUSD) in 1 987 to 
improve the students' academic achievement in 1 0 low-achieving urban schools with a 
predominant African American student population (Appendix A- Reference Guide No. 

10, 1996-97). However, during recent years the TSP population has changed and is 
currently over 50% Hispanic. 

The Office of Student Integration Services of the LAUSD contracted the Evaluation 
and Training Institute (ETI) for an evaluation of the TSP in conjunction with the Program 
Evaluation and Research Branch of the LAUSD. The main objectives of this part of 
evaluation were to: 1) compare the achievement of the TSP students with the 
achievement of students in comparable schools, and 2) identify the instructional 
components of the TSP schools that impact the TSP students' outcomes. This report deals 
only with the first evaluation question by comparing the level of students academic 
achievement in the TSP schools to those of the comparison schools. 

The original evaluation design of the Ten Schools Program (TSP) required selection 
of a set of 10 matched comparison schools. The subsequent evaluations indicated that 
while the Ten Schools Program schools were improving year after year, the “Comparison 
Schools” students’ outcomes were actually declining to the point that they were no longer 
a proper comparison group for the TSP program evaluation (Maddahian, 1 999, 
Maddahian, Pike, and Weisbender, 1996). This judgment led to the selection of a new set 
of 12 schools to match the original 10 schools and two more additional schools in the Ten 
School Program. Program Evaluation and Research Branch (PERB) staff selected this 
new set of comparison schools based on a number of criteria including student academic 
achievement level, school's ethnic mix, and geographical location. Table 1 presents a list 
of the schools included in the TSP, corresponding matched schools, their Academic 
Performance Index (API) in 1999, percent of the school population who are Black, and 
percent of the school population who are Hispanic. 
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Table 1 



Schools Included in the Ten Schools Program and Their Correspo nding Comparison 
Schools 



Ten School Program Comparison School 



School 


API* 


% Black 


% His** 


School 


API 


%Black 


% His 


Bright 


605 


40 


59 


Chapman 


607 


25 


50 


King 


555 


43 


57 


Wilshire Crest 


563 


45 


42 


93 rd Street 


506 


32 


67 


Amestoy 


483 


31 


57 


Compton 


482 


42 


58 


153 rd Street 


477 


45 


43 


Flournoy 


464 


48 


52 


Saturn 


464 


34 


64 


Barrett 


460 


50 


50 


Angeles Mesa 


453 


65 


35 


Joyner 


444 


52 


48 


West Athens 


446 


34 


64 


McKinley 


442 


26 


73 


68 th Street 


444 


25 


74 


96 th Street 


440 


34 


66 


122 nd Street 


424 


34 


64 


112 th Street 


424 


34 


64 


109 th Street 


415 


34 


65 


1 1 6 th Street 


420 


50 


49 


99 th Street 


415 


48 


52 


118 th Street 


356 


40 


59 


Grape 


355 


37 


63 



* API: 1999 Academic Performance Index % Black: Percent of school population who are Black. 

** % His: Percent of school population who are Hispanic. 
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